Successor Feature Sets: Generalizing Successor Representations Across Policies

Reflections on Successor Liability

Deep Successor Reinforcement Learning

Learning robust value functions given raw observations and rewards is now possible with model-free and model-based deep reinforcement learning algorithms. There is a third alternative, called Successor Representations (SR), which decomposes the value function into two components – a reward predictor and a successor map. The successor map represents the expected future state occupancy from any g...

Minimal Indices for Successor Search

We give a new successor data structure which improves upon the index size of the Pǎtraşcu-Thorup data structures, reducing the index size from O(nw) bits to O(n logw) bits, with optimal probe complexity. Alternatively, our new data structure can be viewed as matching the space complexity of the (probe-suboptimal) z-fast trie of Belazzougui et al. Thus, we get the best of both approaches with re...

Successor-Invariance in the Finite

A first-order sentence θ of vocabulary σ ∪ {S} is successor-invariant in the finite if for every finite σ-structure M and successor relations S1 and S2 on M, (M, S1) |= θ ⇐⇒ (M, S2) |= θ. In this paper I give an example of a non-first-order definable class of finite structures which is, however, defined by a successor-invariant first-order sentence. This strengthens a corresponding result for o...

Separation and the Successor Relation

We investigate two problems for a class C of regular word languages. The C-membership problem asks for an algorithm to decide whether an input language belongs to C. The C-separation problem asks for an algorithm that, given as input two regular languages, decides whether there exists a third language in C containing the first language, while being disjoint from the second. These problems are c...